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ABSTRACT 

This paper describes the development of a program to 
test oral and written language skills of students graduating from 
seven Hong Kong higher education institutions. The program*s 
objective is to provide information for prospective employers 
concerning graduates' practical communication skills in Cantonese, 
Mandarin, and English. The 2-year project described here was to 
determine the feasibility of putting such a program in place in 1997. 
The program would assess over 15,000 students per year. Focus in this 
description is on testing of English language skills. The report 
describes the project's background and chronicles progress to date. 
British specialists in language testing were consulted, a report on 
English language teaching in the seven institutions was written, and 
a committee from those institutions was established. The committee 
decided on issues to be addressed in their inquiry: the compulsory 
nature of the testing; nature of the communication to be tested; form 
and substance of int er ins t i t ut i onal collaboration; effect of a 
testing program implemented in 1997; and the consortium's commitment 
to development of assessment and its possible consequences for Hong 
Kong language teaching. Conclusions drawn in these areas are 
outlined. Next steps include reaching a consensus on administration 
of the program, and involvement of the local business community. 
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In April this year (1994) the University and Polytechnic Grants Committee of Hong Kong 
(the UPGC) accepted a proposal from the Department of English of the Hong Kong 
Polytechnic, now the Hong Kong Polytechnic University, to lead an investigation into the 
feasibility of assessing the communicative proficiency of all first degree students attending 
the seven institutions that it funds. This assessment of proficiency in English, Cantonese 
and Potonghua (Mandarin) is to be made towards the very end of a student’s course and 
is to provide prospective employers with an up-to-date, clear indication of what a student 
can do with the languages as that relates to communication in the workplace. The UPGC 
handsomely funded the feasibility study to the tune of HK$ 7m. (or about US$lm.) 



The funding was made available on the understanding that the project should be inter- 
institutional across all the seven, now designated, universities; should cover all three 
major languages of business communication in the territory; and should focus on 
providing employers with "can do" statements about students’ abilities to write and speak. 



UPGC’s purpose in sponsoring the feasibility study, (apart from the ostensible one of 
providing employers with useful, valuable information on individual prospective 
employees), is to use such an assessment (should it be put in place) as one more tool 
(among the many it has developed over the last few years) with which; 



1. to attack the commonly perceived problem of the declining standard of 
English in the colony’s workforce; 
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2. to promote skills in Potonghua’ which is seen as of rising importance in 
the colony’s development; 

3. to focus attention on the need to raise generally the standard of native 
tongue skills in line with the changing requirements of the Hong Kong 
business community. 

One thing to be borne in mind right from the beginning of the study has been that the 
end-user of the assessments will be the business community and not language-orientated 
academics. The assessments should, therefore, not be of; 



According to the 1991 censUvS Potonghua is spoken by only 1.1% of the population of Hong Kong 
as a fiTvSt language and as a second by only 17%. 
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attainment of syllabus objectives, 
potential for improvement, nor 
knowledge of the languages. 



2 . 

3. 



The assessment should tell the employer what an individual, prospective employee can do 
now with the languages, in terms of Oral Interaction and Writing, that is relevant to the 
workplace. 

Should such an assessment become, in one way or another, obligatory for all first degree 
students, and should the quite proper demand from the student body to be prepared for it 
be met, these two factors taken together would radically alter the current balance of at 
least the final year of degree studies for almost all those undertaking first degree courses 
in Hong Kong. The ability to communicate effectively in more than one language would 
become, quite rightly in our view, a major focus for all first degree academic 
programmes. The changes in the university classroom that will be required to align to 
this focus will be fundamental. 

This two year project has been set-up initially to examine the feasibility of establishing 
the "machinery” to allow an assessment of the language skills of final year students to 
take place in late Spring of 1997. The number of graduating students is likely to be in 
excess of 15,000. 

In this short paper we can only describe briefly what has been done to develop a 
framework for the testing machine and explain some of the issues that this project has 
brought to the surface and to offer suggested ways of addressing them. This presentation 
concentrates on the work done to date on the English language strand of GSLPA. Work 
on the Chinese languages strand, which is now well under way, is being conducted under 
the direction of Professor Cheung Yat-Shing, Head of the Polytechnic University’s 
Department of Chinese, Translation and Interpretation and will be reported on separately. 



Part 2 

So what has been done so far? After some ground clearing by way of hard-thinking and 
the sketching of some large and complex diagrams of implications inherent in the project, 
we visited all the departments, institutes and centres that provide ELT for degree students 
in Hong Kong for briefing/ fact-finding sessions. 

Last summer we were also lucky to be able to visit various language testing specialists in 
the United Kingdom to tap their thoughts. 

Following on from this, we did two things: one was to write up a report on ELT 
provision in the seven institutions; and the second was to set up a working party made up 
of members from those institutions. 
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The Working Party came up with this list of issues that would need to be addressed: 

1. The compulsory nature of the assessment 

2. The nature of the communication to be tested 

3. The form and substance of the collaboration between the UPGC institutions 

4. The effect of an in-place test of language proficiency at the end of 

undergraduate programmes in 1997 

5. The UPGC commitment to the development of the assessment and its 
possible consequences for language education in Hong Kong. 

These are of course net entirely separate issues. They overlap and intertwine and feed off 
each other. 



Part 3 

As for the compulsory nature of the assessment as contained in the original proposal, the 
feeling of the Working Party was that ic would be difficult to make the assessment 
compulsory for all graduating students in Hong Kong at this stage, for a variety of 
reasons. There could probably be objections from the student body relating to the 
additional burden of the assessment without a full perception of its need in the context of 
their ov/n career development.^ 

However, as Tim MacNamara pointed out during the working party, this issue was way 
down the line and it was concluded that; 

t/ie compulsory nature of the assessment should be re-considered at a later stage in 
the development of the GSLPA project: but that the end-users of the assessment, 
the employers of graduates in Hong Kong, would be the major inspiration for the 
adoption of the machinery that would ^ it the assessment in place. 



As we thought through this, it became clearer that the involvement of Hong Kong’s 
Captains of Industry in this exercise will need to be great and probably take place at all 
stages of the project development. In particular when we consider the nature of the 
communication to be tested. (See issue 2, above.) 

In addition to student objections to additional assessment, it was noted that the English 
Language Teaching operations in the seven tertiary institutions varied considerably. 
There were variations in the length of the ELT offered, variations in the number of 
students taught and in the subject disciplines targeted for ELT. In many cases English 
tuition for students ceased after the first academic year. The students sitting for the first 
administration of the assessment in 1997 would be that cohort who started their tertiary 
education in October this year. The majority of these students will have ceased their 



^ The working party wondered whether the assessment would take into account the number of non- 
native Cantonese speaking undergraduates at the various institutions. 
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English classes by 1995 - two years before the assessment might be in place. It was 
therefore concluded that: 



if the present FFL operating systems and practices continue in the various UPGC 
institutions with no acknowledgement of the fact that language proficiency 
assessment may be in place by the time the present cohort of undergraduates 
graduate, then the assessment would: 



a) disadvantage those students who had not had EFL courses up to the end of 
their third year of study. 

b) make the construction of the assessment impossibly complicated and 
cumbersome taking into account the varying nature of the courses taken by 
students. 

c) not give fitture employees of Hong Kong graduates an accurate description 
of students’ language proficiency. 



This is not simply an oblique reference for more UPGC funds: it has become increasingly 
apparent in Hong Kong that throwing money at teachers doesn’t necessarily help to 
improve students’ language skills. It raises another issue - that of more fully integrating 
language teaching operations in the universities with the main objectives of academic 
programmes. It seems at present that English for Academic Purposes and the "Business 
English" courses are often tagged on at the end of a degree course document and English 
teachers are required to perform miracles in one hour week. A stronger motivation from 
within the institutions is maybe required so that, for example, there is at least timetable 
space for us to work in. 

This "issue" has certain input to the next consideration which The Working Party noted, 
this being the effect that assessment of language proficiency would have on the nature of 
EFL courses at the various institutions, particularly as the proficiency would be related to 
"language for business and commerce". The wash-back effect would necessitate a "teach- 
to-the-test" practice in class rooms with a proliferation of "practice test" papers and 
books. While not necessarily a negative educational philosophy, this "test-driven" 
approach to language teaching could effectively deny the current degree of institutional 
autonomy in EFL course design, particularly where EFL courses are designed specifically 
to support the English language needs of specific degree courses 

GSLPA will have to be designed with an eye to current teaching practice and 

course objectives within each institution’s language teaching operation. 

This point is taken up later in this paper as a problem that relates to the management and 
administration of GSLP.\ in Hong Kong. 
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The \'/orking Party noted that, in an ideal world, the assessment as currently proposed, 
would have a dramatic and positive wash-back effect on the improvement of language 
proficiency and language teaching practices. 

The assessment, when in place and compulsory would, for reasons of 
accountability to society, require the provision of funds to cover the need for 
language education for the three years proceeding assessment for each and any 
cohort of graduating students. This funding would include, as mentioned, buying 
time and space for English language teaching 



The provision for such funds, that would allow three full years of language teaching at all 
institutions for all undergraduates, would probably not be forthcoming for the current 
term of the GSLPA Project. If such a test were in place and compulsory in 1997, those 
students undergoing assessment (the current first year undergraduates) would be 
disadvantaged, as mentioned above. 

The assessment might better be seen as a pilot test up to and possibly after 1997. 
As such assessment would not be made on all graduating students, initially, the 
development of assessment might be better targeted at graduating students from 
perhaps a few institutions, with cooperation in trialling from other institutions. 
This would give more time for the development of a reliable and acceptable test - 
given the current disparate EFL provisions, and would give time for the 
de'’elopment and finding of language courses that were more precisely related to 
the aims of the GSLPA. 



Part 4 

The complexity of the above constraints and the insights we have gathered on the 
practicality of the project have brought us to a second planning stage from which we hope 
to move forward quickly. At this point we feel that it is important to gain some form of 
consensus quickly on the management and administration of GSLPA across seven 
institutions with different histories, student populations and academic structures. To 
address this need three models for the management and administration of GSLPA have 
been devised and examined for practicality given the complex situation into which the 
assessment would need to fit. 

The first, a traditional "public examination" type of test administered annually to all 
graduating students, really needs no gloss. The well known characteristics of these types 
of test formed the basis for the discussions of our first working party and raised the issues 
as discussed. 

The second model started off with a flippant comment by a colleague, but as we hope you 
will agree once developed is not without some merit. The comment was: 

The job market in Hong Kong being what it is, most graduates in Hong Kong have 
no difficulty getting a job anyway so why bother to assess their language skills? 
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Well - model two takes this premise at its face value. All students should be tested but 
not assessed. The samples of texts gathered from students could be stored and only 
assessed when a prospective employer or the student concerned requests the service. 

The "service" of assessment could be charged for and take either the form of a simple 
grading (cheap) or a consultation based on the sample of texts that have been gathered 
(expensive). Alternatively, the samples of texts could be made available to prospective 
employers for them to make up their own minds on whether the pwDtential employee has 
the language skills that are required (cheap). This last alternative has the same rationale 
that many professions adopt through requiring practitioners in a particular field to 
maintain a portfolio of their work. 

What started off as a rather light-hearted idea, as you can see, taken a little further does 
actually start to coalesce into a coherent model 

The third model does away with the idea of a monolithic annual test sat by all students in 
their final year and substitutes for it a series of final year assessments made locally in 
each institution. 

The task of standardising these assessments would be carried out by a proactive group of 
language assessment specialists drawn from the participating institutions and responsible 
for the following: 

1 the initial provision to participating institutions of; assessment specifications, 
exemplars of item types, criterion texts, exemplar marking schemes, descriptors 
for reporting GSLPA 

2 external validation of; the tasks set students to elicit performance upon which 
GSLPA assessment is made; the criterion texts used in assessment; marking 
schemes 

3 external moderation of a sessments. 

A monolithic, exam-based model of GSLPA would probably at best generate 30 minutes 
of individual student Oral Interaction and one hour of Writing on which to base a 
description of communicative proficiency. This alternative model would be more than 
preferable if it only generated double the quantity of student performance on which an 
assessment could be made. 



Part 5 

The final issue which perhaps underlies all the above is, of course, what happens after 
1997? 

Already the development of English language proficiency testing is taking place in the 
People’s Republic of China with the help ot international consultancies mainly from the 
dK. In particular the University of Guangzhou with the University of Cambridge Local 



Examinations Syndicate (UCLES) and the London Chamber of Commerce (LCC) are 
developing a test of Business English for undergraduates in Guangdong Province. 

It seems sensible that the GSLPA project team should collaborate with the Chinese 
test developers and in as far as it is possible or necessary with UCLES and LCC, 
so that the aims and objectives of proficiency testing on both sides of the border 
are similar such that when there is no border the GSLPA may be a suitable system 
for the assessment of language proficiency for graduating students from 
universities in southern China including those from the special administrative zone 
of Hong Kong. 



So what happens next? 

1 . Consensus on the form of the management and administration of GSLPA within 
Hong Kong needs to be attained. 

2. The approval and active participation of the local business community needs to be 
sought to generate the impetus necessary within the community of higher education 
decision makers, the student body and the community in general; so that this most 
important project bears the fruit for which it was seeded - dramatic and pervasive 
change in the university classroom that aims at placing multi-lingual 
communication skills at the heart of all higher education programmes. 
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